Refining the Utility Metric for Utility-Based Cache Partitioning

It is expected that future high-performance processors will implement large L2 or L3 caches that will be shared by multiple cores. Allocating shared cache space among multiple programs is an important problem and has received much attention in recent years. Typically, the ways or sets of a cache are partitioned across multiple competing programs. Most recent work is built on the concept of marginal utility, i.e., a way is assigned to a program that is expected to benefit most by receiving that additional way. Benefit is usually quantified as misses per kilo-instruction (MPKI). A cache partition is chosen such that overall MPKI for a workload is minimized. However, the ultimate performance metric of interest is overall IPC or throughput, not overall MPKI. MPKI is used as a proxy for IPC because it is much easier to compute – recent work has suggested that MPKI per additional way can be easily computed by maintaining a small shadow tag structure. In this paper, we first quantify how the use of MPKI instead of IPC can result in sub-optimal cache partitions. It is wellknown that misses have varying impacts on IPC across programs because of varying levels of latency tolerance in each program. As a result, we discover a non-trivial number of cases where the use of MPKI is sub-optimal. This number increases as more programs share a given cache. We then propose a simple mechanism that uses two IPC samples and a miss rate curve to compute an IPC curve. This allows us to better quantify marginal utility in terms of IPC and arrive at performance-optimal cache